ENH: JupyterLite integration for the documentation - #13925
ENH: JupyterLite integration for the documentation#13925natinew77-creator wants to merge 146 commits into
Conversation
1e40137 to
f5bbad2
Compare
|
Hi @teonbrooks, Status Update: This PR is now ready for your review! As the first step in my GSoC roadmap, this PR successfully introduces the core JupyterLite infrastructure to the MNE-Python documentation. Here is what has been achieved:
I also looked deeply into the To gracefully handle this and prevent user confusion, I've written a custom For smaller datasets that are CORS-friendly, it automatically falls back to Pyodide's native I think this is the best architectural approach for handling the massive tutorials. I'm ready to mark this PR as complete so we can move down the GSoC checklist and start tackling |
|
Quick Follow-up: It turned out to be a race condition during the Sphinx |
10c4d98 to
c41059a
Compare
Integrates jupyterlite-sphinx into the MNE-Python doc build so every sphinx-gallery example gets a 'Try in Browser' button backed by a Pyodide/WebAssembly kernel. - doc/conf.py: configure jupyterlite_sphinx; build a local MNE dev wheel with relaxed Pyodide constraints; copy required MNE sample-data subset into JupyterLite's virtual filesystem; inject a setup cell that installs MNE via micropip (keep_going=True bypasses version conflicts), mocks missing stdlib modules (lzma, multiprocessing), patches pooch to block large OSF downloads, and sets MNE_DATA paths - .circleci/config.yml: ensure MNE sample data is on disk before the doc build so conf.py can copy it into jupyterlite_contents/ - .github/workflows/jupyterlite.yml: standalone GH Actions workflow on the jupyterlite-gh-actions branch that builds and uploads the site - pyproject.toml: add jupyterlite-pyodide-kernel and jupyterlite-sphinx to the [doc] extras - .gitignore: exclude jupyterlite_contents build artifacts
- mne/parallel.py: return False early in _running_in_joblib_context() on emscripten; joblib parallel backends are unavailable in the browser - mne/utils/config.py: catch Exception (not just ValueError) when loading the MNE config JSON; Pyodide's json parser raises SyntaxError on a corrupt or absent config file
Tutorials and examples that call interactive Qt backends (raw.plot(), epochs.plot(), ica.plot_sources(), etc.) or depend on large datasets not bundled in JupyterLite will hang or error in Pyodide. Wrap them with sys.platform guards so they are skipped when running in the browser. Interactive Qt plots (skip on emscripten): - tutorials/intro/10_overview.py: raw.plot(), stc.plot() - tutorials/intro/15_inplace.py: original_raw.plot(), rereferenced_raw.plot() - tutorials/intro/20_events_from_raw.py: raw.copy().pick().plot(), raw.plot() - tutorials/intro/40_sensor_locations.py: mne.viz.plot_alignment() - tutorials/evoked/40_whitened.py: raw.plot(), epochs.plot() - examples/preprocessing/muscle_ica.py: all ica.plot_* calls Large datasets unavailable in the browser (raise RuntimeError on emscripten): - tutorials/io/60_ctf_bst_auditory.py: BST auditory dataset (~2.9 GB) - tutorials/io/70_reading_eyetracking_data.py: EyeLink misc dataset - examples/visualization/eyetracking_plot_heatmap.py: EyeLink dataset
f0095cf to
9e6e288
Compare
…erLite - doc/conf.py: Fix lzma mock to use real stdlib lzma when available in Pyodide instead of LZMAFile=object which broke joblib's compressor registration - 10_overview.py: Guard ica.plot_properties() which opens an interactive Qt window - 15_inplace.py: Guard set_eeg_reference block which fails under Python 3.13 in Pyodide - 20_events_from_raw.py: Guard STIM channel plot and all EEGLAB sections that require the unavailable testing dataset - 40_sensor_locations.py: Guard ssvep dataset loading and sphere plot that require the unavailable ssvep dataset - 50_configure_mne.py: Guard KIT test data loading whose test files are stripped from the Pyodide wheel - 70_report.py: Skip Report.save() file-writing in browser, guard nibabel-dependent add_bem, 3D methods (add_trans/add_stc/add_forward/ add_inverse_operator), missing ECG/events files, pandas-dependent make_metadata, and the HDF5 round-trip section All intro tutorials (10, 15, 20, 30, 40, 50, 70) now run cleanly in JupyterLite/Pyodide without errors. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
This reverts commit 60fcc7f.
This reverts commit 825740e.
set_3d_view and the other scene-level helpers read renderer.backend directly rather than going through _get_renderer, so they were still calling into None.
It asks make_field_map to subdivide the helmet mesh through VTK, and plot_field needs the interactive viewer, so neither blocker is reachable from the browser.
plot_alignment and plot_bem locate their surfaces by probing the filesystem before any reader runs, so the browser has to fetch the candidates up front. Also hides the launch badge on 14 notebooks that need the Brain widget layer or data too large to bundle.
read_raw_nirx and read_raw_egi are handed a folder rather than a file, so the copy leaves a manifest beside it for the browser to walk. Files past a size limit are skipped rather than bundled, and the seven notebooks that need fsaverage, mne_bids or a multi-hundred-MB volume model are excluded instead.
…rcle full] The curated lite_data archive only carries the files it was published with, so anything added since went missing from the browser bundle without failing the build. Prefer whatever CI already restored and fall back to the archive.
The tutorials no longer branch on sys.platform: the data those blocks skipped is served now, and reading a KIT file or a BrainVision header goes through the setup cell like everything else. 70_report keeps too much it cannot render, so its badge is hidden instead.
The full build reports 379 MB and 251 MB for these, so the copy step skips them and the badge would have had nothing to load.
Its raw alone is 344 MB, 404 MB with the forward and surfaces, which is more than those six pages are worth on every docs deploy.
|
PR status comment The 31 files that showed a badge but failed are down to zero. 53 pages have the button hidden, 153 have it and their data actually loads. The sys.platform guards are gone too, so every tutorial and example is byte-identical to upstream now and all the browser code sits in doc/conf.py. 3D now covers plot_alignment, plot_bem and source estimates through pyvista-js. Brain and plot_field still don't work, so those pages are excluded. Dropped somato after checking with Teon, 404 MB for six pages was a lot to ship. |
Most MNE readers validate a path through _check_fname before opening it, so hooking that covers read_info, read_evokeds, read_cov and the rest at once; read_label, read_epochs and read_raw_edf open directly and are wrapped individually. Also serves talairach.xfm, the auditory/visual labels and the EEGBCI runs CI provides, and installs mffpy and python-picard.
… full]
dig_mri_distances goes through mne/surface.py rather than _freesurfer, so the
existing shim never fired. setup_source_space also needs surf/{lh,rh}.sphere.
An oct-6 source space meant 8196 meshes and 8196 actors in a single scene, which ran the browser tab out of memory. Also honor glyph_resolution/height and solid_transform, so the MRI fiducials come out 5 mm rather than a metre.
The tutorials that build a renderer themselves reach for renderer.figure, which the pyvista backend exposes alongside scene(). Honour a fig argument too, so plot_alignment and plot_dipole_locations composite into one scene instead of opening a second one.
…e full] mne.gui.coregistration places fiducials by clicking the scalp, and the vtk.js renderer has no picker, so that one cell cannot run. Swap it for a note via sphinx-gallery's notebook_modification_function, which only touches the JupyterLite copies, and the other nine cells keep working.
…ircle full] conf.py asserts sphinx_gallery_conf is serializable and sphinx rejects function objects, so passing the hook directly broke the build before Sphinx started. sphinx-gallery imports a dotted path instead, so move it to sphinxext.
first_notebook_cell is applied when sphinx-gallery generates a notebook, so the piplite cell also landed in the .ipynb offered for download, where it fails on the first line. Prepend it at copy time instead, alongside the cell notes, so only the JupyterLite copies carry it.
Twelve of the readers only need their file fetched before MNE opens it, and each had its own six-line copy. One wrapper plus a table of readers does the same job; the ones that need more than a fetch keep their own shim.
Tracking Issue: #13929
What does this implement/fix?
This PR integrates JupyterLite into the MNE-Python documentation build, allowing users to run tutorials interactively directly in their browser without a local Python environment.
Key technical implementations:
jupyterlite-sphinxinto the Sphinx-Gallery pipeline, automatically generating "Try in JupyterLite" buttons for tutorials and examples.conf.pyto automatically handle Pyodide-specific browser quirks:mneandpyodide-httpnatively viamicropip.pyodide_http.patch_all()) so MNE'spoochdownloader can successfully fetch datasets from the browser.mne.viz.utils.plt_showto correctly render MNE's Matplotlib figures inline within the WebAssembly environment.Additional information
mainbranch version mismatches since JupyterLite currently pulls the stable MNE release.